The present invention relates in general to automated kinematic machines and systems, such as, but not limited to, remote tool/robot control systems, and is particularly directed to a telekinegenesis system for training the sequential kinematic behavior of an automated kinematic machine by means of a virtual reality simulator, driven with kinematic parameter data out from a teleoperational device, which models the sequential behavior to be exhibited by the automated kinematic machine.
In the generalized field of robotics, which is defined herein as the remote control and operation, either in response to discrete operator commands or autonomously, of plural function, multiple degree of freedom electro-optic, opto-mechanical, and or electromechanical systems, extant practices may incorporate one or more of telepresence, teleoperation, and telekinesis. In such robotic systems, calibrations, ranging from one-time measurements to periodic measurements of specific tooling points or arrays of known positions, to real time mensurations with concomitant program parameter updates, are commonly employed. Also, devices which utilize telepresence, telekinesis, and teleoperation with various control, operation, and calibration schemes are extensively documented in various patent literature, textbooks, technical publications, industry publications, and contemporary articles in a variety of popular publishing formats, including the Internet.
There are numerous current and future robotics operations, that differ in operating environments, functional sophistication, and criticality of correct and adequate operation. Industrial robots are widely used as automatons, repetitively performing the same sequence through the life of a production run. They may then be retooled and reprogrammed to perform a different set of tasks. Although generation of programs for this type of robotic application is not the prime focus of the present invention to be described below, a brief description will serve to elucidate some of the simple, underlying principles of robotics command program generation, and serve as initial introduction to telekinegenesis principles that are common to extant robotics system practices.
In cases of relatively simple automaton applications, the operating environment, motions required, effects of robot actions, physical calibrations, etc. are very well known, and explicitly definable in simple sequences and geometrical terms. The requisite actions are relatively easy to define in explicit, simple terms, and the command programs can be iterated and calibrated to near perfection. As programs of this sort may be used to generate large production runs, considerable time and money can be practically and economically invested to develop programs that yield the required results, particularly for operations such as pick and place, spot welding, fastener installation in set locations, and other similar functions that require simply definable motion parameters and sequences. Tasks such as these utilize telekinesis; that is, machines that are moved through a series of positions as specified by discrete kinematics commands. These machines employ calibrations with a wide range of sophistication, depending on the precision required.
Development of a command sequence may be characterized as shown in the functional block diagram of FIG. 1. In general, as shown at 101, it is necessary to define the work space or task environment, which is typically updated or modified at 102, with the results of physical calibrations, modifications, etc. End position work points 103 are then mathematically defined at 104, and are translated into joint positions 105 and associated joint position commands 106. These joint position commands, in turn, usually require correction or modification 102 via machine and workspace calibration 107, depending on precision requirements for the operation. Except for the simplest of tasks, or those that are very mature, a validation process 108, with some subsequent modification at 102, is performed. Everything is then updated, and the system is put into operation at 109.
It should also be noted that, for most cases, the end effector position of a robotics assembly is not a unique function of the joint positions. Added constraints of work space envelopes and non interference with work piece and support structures limit the allowable sets of joint positions from which the programmer can then select. As the number of degrees of freedom of the robotics assembly increases, calibration requirements become more stringent and selection of the optimal joint position commands becomes less intuitive, and therefore more difficult for the program developer to specify correctly.
Functionally sophisticated tasks requiring intuitive, adaptive, human like control, such as installing gears in transmission assemblies or clamping bleeders with hemostats and installing sutures in a surgical operation, as non-limiting examples, often utilize teleoperation principles. Non-limiting examples of teleoperational robotic systems are described in the U.S. Patents to Kaneko et al, U.S. Pat. No. 5,341,458, and Aono et al U.S. Pat. No. 5,483,440.
As diagrammatically illustrated in FIG. 2, in a conventional teleoperational system, the operator manipulates a kinematics simulator (or master) 20, shown as having a plurality of joints 1, 2 and 3, the movement of which is sensed by a controller 22, to generate the position commands for a multijointed slave robotic manipulator 24 that actually performs the work. The slave may be in the proximity of the operator, or quite far away. Telepresence of some form is generally incorporated, in the form of remote video, measurement systems, and other sensors. Some teleoperation systems also incorporate tactile sensing and force feedback 26 from the slave 24 to the controller 22, and feedback 28 from the controller 22 to master 20. This feedback serves to provide the operator with some xe2x80x9cfeelxe2x80x9d for what is happening at the slave.
In most applications, teleoperation has the additional benefit that operator-positioning of the robotic kinematics simulator is intuitive, as the operator is simply moving arms where he wants them, not commanding a joint and trying to anticipate where this will position everything. Since the operator is evaluating each move in real time and adapting his actions, work space and task environment definition does not require the mathematical precision of the previous example. Some advanced systems further incorporate some dynamic controls to eliminate natural tremor from the operator inputs, and provide precision position feedback as well.
Although teleoperation systems inherently lack the machine precision of telekinesis systems, they lend themselves quite well to managing functionally complex sequences, or operations where there is no a priori precision in task environment definition. Of course, if the operator makes an error, so does the slave. Sensing and process controls can ameliorate this, but it is still fundamental to the nature of this type of operation.
It should also be noted that, for teleoperation, scale differences between the master and slave can be quite large. A remotely operable crane, for example, can be operated using a small desktop scale model of the real crane. The teleoperational principle works reasonably well for a large number of degrees of freedom, and it may work well for coordinating more than one machine. Experimental teleoperational medical robots, for example, commonly employ both a right hand and left hand manipulating arm, which are operated in concert by the surgeon.
Applications for robotics operations are evolving, which have requirements for hi h reliability, functional complexity, high precision, and with failure consequences that mandate maximal validation during command program development and operation. A few examples of potential applications with these characteristics are conventional weapons disarming, nuclear weapons disassembly, toxic waste inspection and cleanup, and deactivation and cleanup of failed nuclear power plants such as Chernobyl.
These applications share a number of characteristics, such as high degree of functional complexity, coordination of multiple machine actions, task environments that may have unanticipated characteristics, generation of command programs and sequences with a very short or even real time development cycle, and failure consequences that mandate high fidelity operational validation with limited or no practice available. They also feature task environments that are either extremely hazardous or lethal to human operators, which mandates the use of robots, and reliance on remote measurements and calibration to update operating parameters in real time.
In accordance with the present invention, these needs are successfully addressed by what is referred to herein as a xe2x80x98telekinegenesisxe2x80x99 robotics command program generator and validation methods therefor. The telekinegenesis system of the invention involves a method and architecture for generating robotics system command programs, by combining the quantitative precision of traditional telekinesis with the intuitive, adaptive, sophistication of teleoperation, to realize a development and validation method that drastically reduces the time required for generating precise command and control, in combination with sophisticated kinematics coordination.
As will be described, the telekinegenesis system of the invention is capable of generating coordinated multiple degree of freedom mechanism motion sequences, spatial relationships for sequential kinematics parameter generation, coordination of sets of multiple degree of freedom robots, singly and in combination, as well as complete robotics command programs by concatenating measured kinematics simulator outputs with available robot command program generators. It may further include, singly or in combination, extant calibration methodology including a priori and real time methods, virtual reality simulation, and instruction modification by artificial intelligence system design tools such as ICAD and other similar tools.
The telekinegenesis system of the invention may also employ neural learning techniques, and may additionally incorporate, singly or in combination, modifications to the task environment model and operating constraints via remote measurement of the physical task environment. This may be accomplished either by sensors and measurement capabilities embodied in the robotics system, ancillary equipment, or measurement systems unrelated to the robotics system employed, with modification of the task environment model and subsequent modification of the robotics command set to adapt to the physical task environment, which may be different from the task environment model that was used to generate the initial set of commands.
Advantageously, using the telekinegenesis methodology of the invention, programs may be developed well in advance of the actual robotics operation, or may be used to generate sophisticated, program sequences during the actual operation itself with validation via virtual reality simulation, comparison of generated command positions with physical data, and analyzed for compliance with mission tenets, operating constraints, and system rules via artificial intelligence analysis of the generated sequence.
According to the functional operation of the telekinegenesis system of the invention, joint parameters of an operator-controlled simulator are sampled by a computer, and formatted for input to a robot program generator. By virtue of the simulation, the joint parameters are coordinated, and the robot program generator outputs a set of instructions required to command a robot to perform the same actions. This program is downloaded to a robot controller, which then executes the task.
In a practical implementation, the telekinegenesis system of the invention employs a virtual reality simulator that is driven with kinematic parameter data derived from a teleoperational device, which models the sequential behavior to be exhibited by the target machine to be controlled. The virtual reality simulation guides the end user through a menu-driven series of selection options and input requests that define the required work cell operation in an intuitive manner, with easily understood requests for quantitative inputs. The teleoperational device contains a plurality of geometrically distributed sensors mounted on a multi-axis adjustable, operator-manipulated leader unit. As the leader unit is manipulated to follow a desired travel path, the sensor outputs are processed by a geometry conversion algorithm executed by the teleoperational device, thereby generating data representative of the spatial kinematics of the desired travel path.
Where the software employed by the teleoperational device allows customization of the format of its output data, the data is preferably formatted so as to be directly interfaced with a kinematic machine simulator program installed in a robotic control simulation workstation. Should such data format customization not be afforded, the data is translated by means of an intermediate format translation algorithm to a format compatible with the simulator program.
The kinematic machine simulator program within the robotic control simulation workstation is configured to simulate a virtual machine based upon the actual design parameters of the target machine to be controlled. Thus, when executed, the simulation program presents the operator with a virtual reality, off-line representation of the target machine, whose on-line behavior is to be controlled. The virtual reality simulation program is interactive. As a consequence, in the course of monitoring the machine""s dynamic behavior, which mimics he spatial kinematics of the travel path, the workstation operator is able to selectively interrupt the operation of the machine, modify its control parameters, and then rerun the program, to the extent necessary, until the desired behavior of the target machine is achieved.
Once the target machine""s simulated kinematic behavior produced by the virtual reality simulation workstation exhibits the desired dynamic spatial geometry profile, customized spatial parameter data stored in the workstation is processed by the virtual reality simulation program, to produce a sequence of kinematic control instructions that are to be downloaded into the microcontroller of the target machine. These off-line sourced instructions, once downloaded into the target machine, are operative to cause the machine, when installed into an operational robotic system, to exhibit its intended on-line sequential kinematic behavior.